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(57) Abstract: A peer review system for scholarly texts is provided The system includes a computer (12), a database of reviewers 
(40), and a program for retrieving a list of reviewers corresponding to the content of the text. The reviewer database (40) preferably 
includes information concerning an expertise of each reviewer. In another aspect, the invention provides a research retrieval system 
comprising a database of peer-reviewed articles indexed for retrieval with category information. The peer-reviewed articles may be 
published together with updates and letters, both of which may also be peer-reviewed. In another aspect, the invention provides a 
system for organizing a conference for presentation of scholarly research. 
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Trtleofthe invention 



ELECTRONIC PEER REVIEW AND PUBLICATION OF SCHOLARLY WRITINGS 

Field of the Invention 

5 The invention relates to a system for electronically conducting and 

managing the peer review process for distribution of scholarly research, and 
more particularly, to a system for improving the processes for both publishing 
articles as well as organizing conference presentations. 

Background of the Invention 

1 0 Peer review of scholarly research prior to publication or presentation is 

widely considered to be the best way to achieve high quality of content in 
journals and conferences. The reputation and reliability of a scholarly journal or 
conference depends on the intellectual quality of its content Peer review, 
however, is a very time consuming process with the consequence that important 

1 5 research may not be journal published for up to two years after its completion. 

Several sources of delay exist in the peer review and journal publication 
process. The first is delay in obtaining the peer reviews. Reviewers are by 
definition leaders in their chosen field and not only have many demands for their 
time and attention, but also are typically engaged in their own very important 

20 research projects. One reason for delay is simply a lack of reviewer time. They 
often receive large volumes of mail, including numerous articles/presentations to 
review. Accordingly, delays may also be caused by the mere fact that reviewers 
have lost, misplaced or left behind in their office the papers to be reviewed. 
Management of the peer review process by editors through the mail is therefore 

25 equally difficult 
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A second source of delay is the actual printing and distribution of the 
journal. A journal issue can not be printed until every article to appear in the 
edition has completed the peer review process. In this regard, a single slow 
reviewer may effectively delay publication of as many as 1 0 other papers. Once 

5 all the articles are complete, the journal must be laid out citations checked, 
indexes and tables of contents generated and the like. Upon completion of the 
set up and proofing work, time must be reserved at the press location for 
printing/binding. Journals are virtually never printed daily, but rather are printed 
periodically, e.g. monthly or quarterly. Delivery is usually by book rate to save 

1 0 money at the expense of additional delay. Problems with delay in publication of 
peer-reviewed articles are particularly acute in fields such as medicine where 
early distribution of research could be a matter of life and death. 

Conferences have been organized so that abstracts or posters of a 
research project may be disseminated before full publication of articles. With 

1 5 quick dissemination, however, come risks to the conference, the author and 
those who would rely on the research that has not yet been fully peer reviewed. 
Since presentations are abstracted or excerpted from full papers they may omit 
critical details. Conferences serve an important function by permitting quick 
dissemination coupled with the ability to directly question a peer about his work 

20 with the possibility that cross fertilization may enrich one's own work. 

Conferences, however, are not a substitution for a complete, peer-reviewed 
descriptions of a research project And, indeed, the system for organizing a 
conference agenda suffers many of the same timeliness difficulties as for journal 
publication. 

25 Another Uisdiivairtdge with published journal distribution of peer reviewed 

research is the difficulty in locating a published paper of interest Learned 
individuals typically subscribe to several journals directed to their area of 
expertise. However, articles of interest to them and their research may be 
published in other journals. The only way for them to identify and obtain access 

30 to these articles is by searching published digests of journal articles, such as 
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Chemical Abstracts . However, to the extent digests are even available for the 
journal of interest they are not compiled and published until well after 
publication of the journal itself, extending the effective period of delay between 
completion and distribution of the research. Libraries also face issues such as 
5 the cost of subscribing to and storing journal subscriptions. 

Online systems, such as MEDSCAPE, alleviate some of these problems, 
such as the cost of storage, but since this and other online systems typically 
only distribute either 1) electronic images of articles previously published in 
journals, or 2) non-peer-reviewed articles, they offer no solution to speeding up 
1 0 either the peer review process or the distribution of peer-reviewed research. 

What is desired, therefore, is a system for speeding up the peer review 
process and for speeding up the distribution of peer-reviewed research. An 
improved system for timely identification of peer-reviewed research of interest is 
also desired as is a system for improving the efficiency of compiling a 
1 5 conference schedule. 

Summary of the Invention 

It is, accordingly, an object of the invention to provide a system for 
improving the speed of peer review and/or publication/presentation of scholarly 
writings. 

20 Another object of the invention is to provide a system of the above 

character in which the peer review process is conducted and managed 
electronically. 

A further object of the invention is to provide a system of the above 
character for electronically assigning, reassigning and managing editors and 
25 reviewers. 
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Yet another object of the invention is to provide a system of the above 
character in which a submitter may electronically resubmit a scholarly writing to 
correct defects identified in the editing process and obtain publication approval. 

Still another object of the invention is to provide a system of the above 
5 character in which a submitter may electronically submit a scholarly writing to 
supplement a prior published article. 

Yet a further object of the invention is to provide a system of the above 
character in which reviewers and/or editors/organizers categorize submissions 
with a system tailored to the content of the data to improve the accuracy of 
10 retrieval by end users. 

Still a further object of the invention is to provide a system of the above 
character in which a conference organizer can select abstracts/posters based 
upon the category information. 

Still yet another object of the invention is to provide a system of the 
1 5 above character in which a user can pay a fee to have a comment submitted for 
review and possible publication in response to a published article. 

These and other objects of the invention are achieved by provision of a 
scholarly research peer review system comprising a text, which may be an 
article or presentation for peer review; a computer for receiving the text; a 
20 database of reviewers accessible by said computer; and a program executing 
on the computer for retrieving a list of reviewers corresponding to the text and 
for forwarding the list of reviewers to an editor/organizer for consideration of 
publication/presentation. 

The reviewer database preferably includes information concerning an 
25 expertise of each reviewer. The program preferably assigns an identifier to the 
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text and stores the text and text identifier on a data storage device. Preferably, 
the computer is connected to the Internet 

In another aspect, the invention provides a peer-reviewed research 
retrieval system comprising a computer; a publish database accessible by the 

5 computer for storing a plurality of peer-reviewed articles; category information 
stored on the publish database together with each of the plurality of peer- 
reviewed articles; a category search request received by the computer from a 
user, and a program executing on the computer for retrieving an article from said 
plurality of peer-reviewed articles which includes category information 

1 0 corresponding to the category search. The category information preferably 
rnctudes gr setee B o rr f re m among at feast basic science and clinicalstudy 
choices. 

Preferably, the category information also includes a selection from 
among basic science, clinical study, academic, practice management and 
1 5 historical choices. Most preferably, the category information includes a field of 
knowledge selection from among the various branches of medicine. 

In another aspect, the invention provides a peer-reviewed research 
retrieval system comprising a computer; a publish database accessible by the 
computer for storing a plurality of peer-reviewed articles, a peer-reviewed 

20 update stored on the publish database together with a corresponding one of the 
plurality of peer-reviewed articles, a search request received by the computer 
from a user, and a program executing on the computer for retrieving an article 
and its corresponding update from the plurality of peer-reviewed articles in 
response to the search request. The system preferably includes a database of 

25 user profiles corresponding to users of the system. The system preferably 
includes a letter received by the computer which pertains to and is stored 
together with a particular article on the publish database. Preferably, the article 
from the publish database includes a description of supplies, and wherein hot 
links to information about the supplies are included in the article text 
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ln another aspect, the invention provides a scholarly research conference 
organizing system comprising a plurality of texts; a computer for receiving the 
plurality of texts, a database accessible to the computer for storing the plurality 
of texts, a program executing on the computer for forwarding a text notice to an 
5 organizer corresponding to a subject of each of the plurality of texts, a reviewer 
assignment input to the computer by the noticed organizer for each of the 
plurality of texts, and a score input to the computer by the assigned reviewer 
indicative of a level of interest in presentation of each of the plurality of texts at a 
conference. 

1 0 Preferably, the corresponding organizer decides whether to present each 

of the plurality of texts ataconference'W^the'aid of the reviewer score for the 
text Preferably, several reviewers are assigned to each text, and wherein the 
organizer makes the presentation decision with the benefit of several reviewer 
scores. 

1 5 The invention and its particular features and advantages will become 

more apparent from the following detailed description considered with reference 
to the accompanying drawings. 

Brief Description of the Drawings 

FIG. 1 is a block diagram of a system for electronically conducting and 
20 managing peer review of scholarly research in accordance with the invention. 

FIG. 2 is a block diagram depicting publication of a peer-reviewed 
research in accordance with the system of FIG. 1 . 

FIG, 3 is a block diagram depicting the editor's management of reviews in 
accordance with the system of FIG. 1. 

25 FIG. 4 is a block diagram depicting the research review process in 

accordance with the system of FIG. 1 . 
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FIG. 5 is a block diagram depicting use of the system of FIG. 1 by an end 

user. 

FIG. 6A and 6B are block diagrams depicting user selection of research 
for review using the category information stored together with the article. 

5 Detailed Description of the Invention 

FIG. 1 depicts a system 10 for conducting and managing peer review of 
scholarly research in accordance with the invention. System 10 comprises a 
computer 12 and group of parties who interact with the computer, namely: 
submitters 14, a managing* etfte* *V6? orttofstogantzefs 18, reviewers 20, and 

1 0 users 22 (see FIG. 5). It is understood, that each of the parties 1 4-20 may also, 
of course, be users of system 10. Further, managing editor 16 may, but need 
not, serve in a dual capacity as an administrator of computer 12. 
Editors/organizers 18 serve as either organizers of a conference or content 
editor's of a journal. In some cases, editors 1 8 may also serve as copy editors, 

1 5 although this is not necessary. It is the interaction of the parties14-22 with the 
computer 12 that forms the basis for the invention. After research has been 
peer-reviewed, the system may be used for article retrieval, article publication 
and supplementation, and/or conference organization. 

Computer 12 may comprise a single computer, a group of computers 
20 such as a server farm, or a network of computers of any type, and parties 14-22 
may interact with computer 12 directly via a console, over a LAN or WAN 
connection, a server and thin client arrangement, or a wired or wireless Internet 
connection. Preferably, the computer is connected to the Internet and includes 
web and mail servers for distributing the various notices, texts, reviews and the 
25 like. A prototype system was implemented using Lotus® Notes® and Domino® 
software, however, it is understood that many other programming languages, 
thin client models and groupware products could also be used without departing 
from the scope of the invention. In this regard, references made throughout this 
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description to functions performed by computer 12 are actually carried out by 
software programs executing on computers. "Programs executing on 
computers" is meant in a broad sense to include firmware and other hardware 
components which include hard-wired logic. 

5 The peer review process begins with submission of a text 24 to computer 

12 by a submitter 14. Text 24 is preferably submitted for review in an agreed 
format and most preferably is submitted on or in accordance with a text 
template 26 which is forwarded to a submitter by computer 12 upon request 
Template 26 includes a plurality of sections such as abstract, conclusion, 
10 method, discussion, results, references, and the like, and category information 
which are common to all submissions. Text 24 is submitted together with a 
submitter identifier which identifies submitter 14. If submitted, the category 
information includes subject information which may be used by the managing 
editor 16 to assign an editor, and/or by the editor 18 to assign reviewers. 

1 5 Upon receipt of a new text 24, computer 1 2 assigns the text a unique text 

identifier and stores the text together with the submitter identifier and text 
identifier in a text database 30. Preferably, text 24 is stored on database 30 in 
sections according to template 26. The text and submitter identifiers allow the 
text and review information pertaining to the text to be quickly and easily 

20 retrieved by participants in the review process. 

Next, computer 12 forwards a text notice 32 to managing editor 16 
notifying her that a new article has been received for peer review. It is 
understood that articles may comprise abstracts or posters for presentation at a 
conference, or complete articles for publication, or third party letters/reviews of 
25 previously published articles, or author's supplements to previously published 
articles. Because articles are published on system 10 electronically, the original 
published article may be appended to include later acquired data in the form of 
updates, reviews, letters to the editor and the like to present a complete, up to 
date, peer-reviewed summary of a research project in a single location. 
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Managing editor 16 must determine whether the submitted text meets the 
publication/presentation standards and editorial policy of system 10 at the level 
of an area of knowledge (see FIG. 6A). Text notice 32 may forward the text 
directly to managing editor 16 for review, but preferably the notice only provides 

5 the managing editor with the text identifier for the article so that the managing 
editor can retrieve text 26 from database 30 and review it at her leisure. The 
managing editor's decision 34 either rejects the text or assigns it to an editor 
based upon the subject matter of the text. It is understood that the subject 
matter of the text may be determined by the managing editor or may be 

1 0 retrieved from the category information supplied by the submitter. 

Computer 12, upon receipt of decision 34 either forwards a rejection 36 
to the submitter by retrieving the submitter identifier stored together with the text 
identifier on database 30, or forwards the assigned editor 18 a notice 38 
including the text identifier. Notice 38 also includes a list of reviewers 20 
1 5 retrieved from database 40 which are knowledgeable about the subject matter to 
which the text pertains. Reviewer database 40 is indexed by expertise of the 
individual reviewers and computer 12 searches database 40 with information 
about a subject of the text which is either submitted with text 24 or is determined 
by managing editor 16. 

20 Notice 38 may include a copy of the text to be reviewed but preferably 

only includes the text identifier so that editor 18 may retrieve text 24 from text 
database 30 at his leisure. The preferred version of this aspect of the invention 
provides the additional advantage that the text to be edited is retained in a 
known place on database 30. The database is accessible to editor 18 as well as 

25 any and all of the assigned reviewers from any computer at any location with, 
e.g., access to the Internet where computer 12 is connected to the internet In 
this regard, loss of, and forgetfulness with respect to the location of, the material 
to be edited/reviewed is effectively eliminated as a source of delay in the peer- 
review process. 
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Editor/organizer 18 makes a reviewer assignment 42 based upon 
matching a subject matter of text 24 with expertise of individual reviews 20. 
Editor/organizer 18 may also give consideration to pre-existing review 
assignment workload and turn-around rates in determining who to assign to 
5 review any new text The number of assigned reviewers and the content of their 
reviews will depend upon the publication/presentation standards and editorial 
policy of system 1 0 at the level of a fieJd of knowledge (see FIG. 6A). It is 
understood that a single editor/organizer 16 may be responsible for one or 
several conferences and/or journals. 

1 0 Upon receipt of reviewer assignment(s) 42, computer 1 2 updates the 

record on text database 30with the assigned reviewers to provide them address 
rights, and increments a project listing for the assigned reviewers on reviewer 
database 40 to update workload. Computer 12 also forwards a review notice 44 
. to reviewers 20 (typically, although not necessarily, there are multiple ones 

1 5 assigned) which includes the text identifier, enabling the reviewers to retrieve 
the text at their leisure. The reviewers may either know from the context or may 
be instructed by the editor in review notice 44 as to the scope of their task. For 
example, if the review notice is for a poster/abstract for a particular presentation 
for a conference organizer or the like, reviewer 20 will know they need only 

20 provide a score 46 for the text relative to other texts they have received. 

Alternately, if the review notice is for a full article, for a journal, for a 
journal editor or the like, reviewer 20 will know they need to provide a complete 
review 48 of each section of the text including the category information and a 
suggested decision on publication. The suggested decision on publication 
25 preferably takes the form: 1) publish, 2) do not publish, 3) publish with this 
correction, etc. 



To facilitate further ordering, score 46 and complete review 48 are 
preferably input to computer 12 by reviewers 20 using a template or agreed 
format Upon receipt the computer updates text database 30 to include the 
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review/score, and preferably also sends a completion notice 50 to 
editor/organizer 1 8 either upon completion of each or upon completion of all of 
the reviews/scores for a particular text Editors/organizers can issue system 
commands to alter completion notice options. For example, the editor organizer 
5 can use notices after each submission to track status of a peer-review process 
and/or can use notices after receipt of all submissions as a tickler to generate a 
consolidated review 52. Consolidated review 52 takes the same form as each 
individual review 48 or score 46, and respectively comprises a combination of 
the reviews or an average score. 

10 Upon completion, editor/organizer 18 forwards consolidated review 52 to 

computer t2 Which stores the review together with the corresponding text on 
database 30. Particularly if it is desirable to save storage space on database 
30, each of the individual reviews 48 and scores 46 may be deleted or archived 
at this or some future time. Computer 12 also forwards consolidated review 52 

15 or a notice relating thereto to submitter 14. 

If the consolidated review 52 includes a decision that text 24 could be 
published if corrected, then submitter 14 has the option to provide the corrected 
information in a resubmission 54 which is stored on database 30 and forwarded 
to editor/organizer 18 upon receipt by computer 12. Following consideration by 
20 editor/organizer 18, a revised consolidated review is prepared and input to 
computer 12 in an iterative process. 

Referring now to FIG. 2, if editor/organizer 18 decides to publish/present 
text 24, then the text is copy-edited by a copy editor 1 8'. It is understood that 
copy editor 18' may be the same or a different person than editor/organizer 18. 
25 When all changes and corrections have been made to the text, copy editor 1 8' 
generates a publish approval 54 which is input to computer 12, The changes 
and corrections made by the copy editor may originate from the consolidated 
review, resubmission, and/or proofreading. Where texts 24 are articles and not 
solely abstracts/posters, and upon receipt of publish approval 54, text 24 and its 
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conresponding category information is moved from a text database 30 to a 
publish database 60 where it is immediately published and made electronically 
available, e.g., via the Internet Upon publication, a publication notice 62 is 
forwarded to submitter 14 by computer 12. 

5 Where at least some of texts 24 are abstracts/posters, and the system is 

intended for organization of a conference or society meeting, publish database 
may be replaced by or supplemented with a conference calendar and/or agenda 
which may, but need not, be published electronically in advance of the 
conference. 

1 0 Referring nev»fe> H&. 3, use of system 16>by an editor 18 to manage the 

peer review process is depicted. As indicated at 64, by presenting an identifier 
(ID) for authentication by computer 12 and an appropriate request, editor 16,18 
can obtain a status report 66. Status reports provide valuable information for 
speeding the peer review process along. The types of data which may be made 

1 5 available include: the reviewers assigned to each text when the assignment 
was made, whether the review and/or consolidated review was completed, the 
number of texts awaiting review by each reviewer, the number of texts assigned 
to each editor, the editors 1 specialties, the reviewers 1 specialties, status of copy 
editing, other presentations scheduled for the meeting, and other information 

20 useful in tracking and managing. 

In response to the status report information, editor 18 can issue a 
reviewer assignment change 68 as needed or desired to manage the peer 
review process. If an assignment change 68 is issued, computer 12 updates 
text database 3Q and reviewer database 40, and notifies the affected reviewers 
25 with a change notice 70. It is understood though not specifically illustrated, that 
a managing editor 16 may use the review process of FIG. 3 in a similar manner 
to change editors. 
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Referring now to FIG. 4, details of the text reviewing process, especially 
where the text is an article for publication, are illustrated. Preferably, computer 
12 includes a personal area, most preferably in web space, where reviewer 20 
can retain his work in progress (WIP). A reviewer's WIP may include a number 

5 of different reviews in varying stages of completion. The reviewer's personal 
area may include links to each review in process, the underlying text for each 
review, dates on which the review was requested and/or is due to be completed. 
The status of other reviewers 1 reviews may also be displayed or available, as 
well as the editor's consolidated review for texts reviewed by the reviewer. By 

1 0 retaining this information on a central computer, it can be more or less 
continuously updated and available for review and continued work by any 
authorized person from any location. 

As indicated at 44 in FIG. 4, the review notice may include an identifier or 
link to the article to be reviewed. In the alternative, however, it is understood 

1 5 that the identifier or link may simply be provided for a reviewer in his personal 
space or computer 12. To obtain access to this personal space or information, a 
reviewer provides a reviewer identifier 72 which may but need not be 
simultaneously input to computer 1 2. Identification 72, which may take the form 
of a usemame and password, is used to authenticate the reviewer and 

20 determine which texts he is authorized to access on database 30. 

If the submitted text identifier, which may take the form of a document or 
key number, matches an authorized text for an authenticated user, then the text 
is provided to reviewer 20 preferably in sections (see 74) matching the format of 
template 26 (see FIG. 1). In addition to the text, reviewer 20 may also obtain 
25 access to the review in progress for the text and possibly other status 
information. 

The review 48 made of the text by reviewer 20 is organized with 
comments and corrections in sections which match those of the text template. 
Review 48 also includes category information supplied by reviewer 20 as well as 
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a recommendation on publication of the text Upon submission to computer 12, 
review 48 is stored on database 30 together with a corresponding one of texts 
24. As described with reference to FIG. 1 , completion notice 50 is forwarded to 
editor 18 as desired. Also, a project list for the reviewer is decremented on 
5 reviewer database 40, so that the reviewer is presumably more likely to obtain 
additional review assignments. 

Editors may receive completion notice via electronic message such as 
email and/or when they connect to computer 1 2. Similar to reviews 20, editors 
18 preferably have personal space on computer 12 such that notice 50 may be 

10 received upon authentication, whether or not the notice was also received, e.g., 
by email. When notice 50 indicates that all reviews have been submitted, editor 
18 is informed that it is time to complete her consolidated review. If notices 50 
are configured for completion of each review, either computer 12 or editor 18 
may keep track of when reviews have been submitted by all assigned reviewers. 

15 If notices 50 are configured for completion of all versions, then its receipt by 
editor 18 is a call to complete the consolidated review. 

To prepare consolidated review 52, editor 1 8 inputs an identifier at 76, 
which may take the form of a usemame and password, for authentication to 
computer 12. The computer uses the identification to determine which texts 
20 among those on database 30 are authorized for access by editor 1 8. In this 
regard, drafts of the consolidate review are stored together with the 
corresponding text 24 and each of the individual reviews. In the case where the 
"editor" is a conference organizer, a conference schedule may be assembled 
and stored in the organizer's personal space. 

25 Editor 1 8 may either select the text identifier or link from her personal 

area or, as indicated on the drawing, submits the text identifier at 76 to computer 
1 2. It is understood that the text identifier may be input together with, before or 
subsequent to the editor identifier. Upon request of an authenticated, authorized 
user, computer 12 permits access by editor 18 to the text and individual reviews 
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78, and to drafts of the consolidate review for completion. Consolidated review 
52 comprises a combination, or a section by section basis, of the comments and 
correction included in the individual reviews as well as those of the editor and 
possibly also those of the copy editor. The consolidated review also includes 
5 the final category information used for indexing and retrieval of the text, and the 
final publication decision of. publish, do not publish, or publish with corrections. 
If the decision is to publish the text, then a notice and/or the text is sent to a 
copy editor in the event editor 1 8 does not herself copy edit the article together 
with the changes and corrections included in the consolidated review. 

10 Referring now to Fig. 5, once a text has been published/scheduled for 

presentation, it is available to be searched and retrieved by users. The system 
preferably provides different rights and privileges to casual and registered users. 
For example, casual users may be able to search and identify, but not retrieve 
articles without becoming a registered user. In this way, a casual user can 

1 5 experiment with system 1 0 and experience first hand the superior search and 
retrieval system explained below with reference to FIGS. 6A-6B. 

The costs of supporting these casual searches may be reduced by 
limiting the number of simultaneous connections to computer 12 and/or by 
selling banner advertisements/links on the pages viewed by the casual user. 
20 These policies may be set and modified by managing editor 16 or another 

authenticated system administrator by issuing appropriate system commands to 
computer 12. Managing editor 16 may also elect not to permit any casual 
searching, but rather to offer an FAQ or demo as an enticement for the casual 
user to register. 

25 Ordinarily, but not necessarily, a casual user 22 may register online 

directly with computer 12 by agreeing to a user policy and providing basic 
information and a credit card for changing retrieved articles. A registration 
template 80, which may but need not include a user policy, is forwarded to a 
casual user by computer 12 upon request Upon supplying the required data, 
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user 22 submits registration 82 for processing. If the registration request is in 
order, a user identifier 84, which may take the forn of a username and 
password, is issued to user 22 for purposes of authentication to computer 12. 
Upon completing a search, a user is presented with the category information of 
5 each 'W and preferably also a brief summary of the text Only if a registered 
user decides to retrieve the full text of the article is he charged. In a library 
setting, payment terms may vary so that, for example, all texts for a particular 
journal are available for full text retrieval on a single terminal in return for a 
subscription rate. 

10 The cost of a registered user's searches may also be offset by 

advertising; butthe comptetetext preferably will not include advertising, but may 
include links to products and manufacturers explicitly mentioned in the text for a 
fee paid by the manufacturer. Most preferably, such a link will not affect or alter 
an original version of the text when printed in hardcopy and is unobtrusive while 

1 5 viewing an electronic version of the text, e.g. a color of the linked words may be 
different 

Citations to the bound version of the text are provided upon payment and 
retrieval of the full version of the article, and may or may not be provided upon 
presentation of search results as set with commands by an authenticated 

20 administrator. It is preferred that the citations only be provided with search 
results in exchange for payments received by the publisher of the bound 
volumes. However, since the electronic version will be available months prior to 
the print version, administrators may elect to present the "citation 11 in order to 
demonstrate that none yet exists and thereby highlight an advantage of system 

25 10. 

Returning to FIG. 5, upon authentication of a user 22 by computer 12 
with user identifier 84, computer 12 presents the authenticated user with his 
personal profile 86. Profile 86 includes personal space on a user database 90 
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for storing search logic, search results, retrieved articles, excerpts of retrieved 
articles, drafts of texts for submission to system 10, and other work product. 

Profile 86 may also provide email, proofreading, abstracting, hard 
copying, binding, indexing, delivery and other services relating document 
5 handling, document management, publishing and distribution. 

An authenticated user may issue commands 92 to computer 12 in order 
to create and organize folders, save, edit and create text, order services and the 
like. In addition to commands relating to profile 86, an authenticated user also 
issues search and retrieval commands in order to obtain texts from publish 
10 database 60. 

Many different types of searching a text database are known and all may 
be used with this invention. Referring to FIGS. 6A and 6B, a particular type of 
category search is disclosed which provides superior results for identification 
and retrieval of peer-reviewed research and especially for medical research. If 

15 no category search is requested, system 1 0 continues at 96 to await a further 
command or request If a category search is requested, a user must first select 
at 98 an area of knowledge 100. In some cases, e.g. an entire web site or 
computer 12 may be devoted to a single area of knowledge 100, such as 
medicine. In other cases, however, it is understood that a single site may serve 

20 as a conglomeration of other sites or as a source of information from multiple 
areas of knowledge. 

Next, at 102, a user selects a field of knowledge 104, such as cardiology, 
orthopedics, etc. where the area of knowledge selected at 103 is medicine, 
where the area of knowledge selected is not medicine, the category search 
25 continues at 1 05. For the remainder of FIGS. 6A-6B, it is assumed that the 
selected area of knowledge is medicine in order to demonstrate how the 
hierarchy of category information is closely tied to content of the texts, i.e. the 
area of knowledge, in order to improve search and retrieval. At 106, user 
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selects a class of report 1 08. For medicine, the classes of report are clinical, 
study, basic science, academic/administrative, practice management, history, 
other. Each of the classes of report are common to each of the fields of 
knowledge. In contrast, each of areas of knowledge take a different set of fields 
5 of knowledge. 

Depending upon the class of report selected, a user is presented with 
numerous additional choices for narrowing his search request which are specific 
to the selected class of report. 

If the selected class of report is clinical study at 1 10, then the choices are 
1 0 type. 112, size 114» design HQ, control population 1 1 8, data. analysis method . 
120 and study population 122. Any number of these choices 112-122, including 
all or none of them, may be selected. 

Options for type choice 1 12 are surgical treatment, surgical technique, 
non-surgical treatment, diagnostic technique, observational/risk factors/natural 

1 5 history, and economic impact/quality of life. Any number of options may be 
selected for the type choice. Options for the size choice 1 14 are case report, 
services of patients <10, 1 1-50, and >50. Only one of the size options may be 
selected. Options for design choice 1 16 are retrospective and prospective and 
only one may be selected. Options for the control population are no controls, 

20 historical controls, concurrent comparison, concurrent nonrandomized and 
randomized. Concurrent non-randomized is only an option for prospective 
designs and randomized is only an option for retrospective designs. Options for 
data analysis are not blinded, single blinded, double blinded, and outcomes 
instrument used. Outcomes instrument is a yes/no check box and a single one 

25 of the other data analysis methods may be selected. Options for study 

population are pediatric, adult, female, male and any number of these may be 
selected. 
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(f the selected class of report is basic science at 124, then the choices 
are subject 126, type 128, and method/source 130, Any number of choices 126- 
130, including all or none of them, may be selected. Options for subject choice 
126 include biology, bioengineering, tissue engineering, anatomy and genetics. 
5 A single selection is permitted among the subject choices. Options for type 
choice 128 include experimental, observational, technique and 
theoretical/modeling. Any number of selections may be made for the type 
choice. Options for method/source choice 130 include human, animal, bacterial- 
viral and non-living, and a single selection is permitted. 

10 If the selected class of report is academic/administrative at 1 32, then a 

type choice may be made- at *34betweef> education and research. 

If the selected class of report is other science at 136, then a type choice 
may be made at 138 between article and editorial. 

Hierarchy 94 represents system for retrieving texts indexed according to 
1 5 category information. In this regard, the category information fixed during the 
peer-review process is used to index the articles for search and retrieval. 
Because the system was designed by medical professionals and customized for 
use with medical content, it provides improvements over so-called "natural 
language" query methods which are not customized for content of the data. 

20 While the invention has been described with reference to a particular 

arrangement of parts, features and the like, these are not intended to exhaust all 
possible arrangements or features, and indeed many other modifications and 
variations will be ascertainable to those of skill in the art As an example of 
several such variations, system 10 could be used to implement an Institutional 

25 Review Board (IRB) and/or an Institutional Animal Care & Use Committee 
(IACUC). IRBs and lACUCs review and approve, reject, or conditioanlly 
approve research programs and/.or treatment options in hospitals and veterinary 
clinics. It is understood that Board/Committee members would take the place of 



SUBSTITUTE SHEET (RULE 26) 



WO 00/65463 



PCT/US00/11188 



-20- 

reviewers while the Chairman would take the place of editors in order to provide 
"peer review" of nonstandard procedures desired to be performed by submitting 
doctors. 
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What is claimed: 

1 - A scholarly research peer review system comprising: 

a text for peer review; 

a computer for receiving said text; 

a database of reviewers accessible by said computer, and 
5 a program executing on said computer for retrieving a list of reviewers 

corresponding to said text and for forwarding the list of reviewers to an editor. 

2. Th& peer review system of ctanrr t wttmuiiT saui reviewer database 
includes information concerning an expertise of each reviewer. 

3. The peer review system of claim 2 wherein said text includes a subject 
identifier and wherein said program retrieves reviewers from said reviewer 
database whose expertise matches the subject identifier. 

4. The peer review system of claim 3 wherein the text is submitted to said 
computer in a template form which includes a subject field for the subject 
identifier. 

5. The peer review system of claim 1 including a text database accessible 
to said computer for storing data, and wherein said program assigns an 
identifier to said text and stores said text and text identifier on said device. 

6. The peer review system of claim 5 wherein said text database includes 
at least one review stored together with the corresponding text and text 
identifier, and further comprising a consolidated review prepared by the editor 
from said at least one review, said consolidated review including a text 

5 identifier and forwarded to said computer by the editor. 
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7. The peer review system of claim 6 wherein said text database includes 
a submitter identifier stored together with the corresponding text and text 
identifier, wherein said program identifies a submitter corresponding to the 
text identifier forwarded by the editor and sends said consolidated review to 

5 the submitter of the text. 

8. The peer review system of claim 7 further comprising a resubmission 
including a text identifier forwarded to said computer, said resubmission 
prepared by the submitter in response to the consolidated review. 

9. The peer review system of claim 8 wherein said program identifies the 
editor corresponding to the text identifier received from the submitter and 
sends the resubmission to the editor for reconsideration of publication. 

10. The peer review system of claim 6 wherein said consolidated review is 
prepared by the editor from a plurality of reviews received from a plurality of 
reviewers. 

1 1 . The peer review system of claim 1 0 wherein said consolidated review 
is stored on said text database together with said text and the corresponding 
text identifier. 

1 2. The peer review system of claim 1 1 wherein said program deletes the 
plurality of reviews from said text database when a corresponding 
consolidated review is stored on said text database for a particular text 
identifier. 

1 3. The peer review system of claim 1 1 wherein said consolidated review 
includes categorization information for indexing said text. 
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14. The peer review system of claim 1 3 wherein each of the plurality of 
reviews include categorization information and a suggested publication 
decision. 

15. The peer review system of claim 14 wherein said consolidated review 
is prepared in sections corresponding to sections of said text. 

16. The peer review system of claim 1 1 wherein said consolidated review 
includes a publication decision. 

1 7. The peer review system of claim 1 6 further comprising a publish 
database for storing texts published on the system, and wherein said program 
switches said text from said text database to said publish database in 
response to the publication decision. 

18. The peer review system of claim 1 wherein the editor selects at least 
one reviewer from the list of reviewers and forwards the selection to said 
computer. 

19. The peer review system of claim 18 wherein said program retrieves an 
address of the selected reviewer from said reviewer database and forwards a 
notice regarding the selection which includes the text identifier. 

20. The peer review system of claim 1 8 wherein the editor selection 
includes an editor identifier. 

21 . The peer review system of claim 20 wherein an editor may present an 
editor identifier and a text identifier to said computer for authentication by said 
program, and wherein the editor may thereafter obtain a status report on the 
peer review process for the text corresponding to the identifier. 
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22. The peer review system of claim 21 wherein the authenticated editor 
may reassign said text for review by another reviewer. 

23. The peer review system of claim 22 wherein said program, in response 
to a reviewer reassignment, prepares and forwards a change notice to an 
affected reviewer. 

24. The peer review system of claim 22 wherein said program, in response 
to a reviewer reassignment, deletes a prior reviewer's identifier from a record 
corresponding to said text on a text database and replaces it with a new 
reviewer's identifier. 

25. The peer review system of claim 1 wherein said computer is connected 
to the Internet 

26. A peer-reviewed research retrieval system comprising: 
a computer; 

a publish database accessible by said computer for storing a plurality 
of peer-reviewed articles; 
5 category information stored on said publish database together with 

each of the plurality of peer-reviewed articles, said category information 
includes a selection from among at least basic science and clinical study 
choices; 

a category search request received by said computer from a user; and 
10 a program executing on said computer for retrieving an article from 

said plurality of peer-reviewed articles which includes category information 
corresponding to said category search. 

27. The research retrieval system of claim 26 wherein said category 
information includes a selection from among basic science, clinical study, 
academic, practice management and historical choices. 
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28. The research retrieval system of claim 27 wherein said category 
information selection is clinical study, and wherein said category information 
further comprises a sub-selection from among size and data analysis choices. 

29. The research retrieval system of claim 28 wherein said category 
information selection is clinical study, and wherein said category information 
further comprises a sub-selection from among type, size, control population, 
and data analysis choices. 

30. The research retrieval system of claim 29 wherein said category 
information selection is clinical study, and wherein said category information 
further comprises a sub-sefection from among type, size, design, control 
population, data analysis, and study population choices 

31. The research retrieval system of claim 27 wherein said category 
information selection is basic science, and wherein said category information 
further comprises a sub-selection from among subject, type and 
method/source choices. 

32. The research retrieval system of claim 26 wherein said category 
information includes a field of knowledge selection from among the various 
branches of medicine. 

33. The research retrieval system of claim 26 wherein said computer is 
connected to the Internet. 

34. The research retrieval system of claim 26 wherein said category 
information includes an area of knowledge selection. 

35. A peer-reviewed research retrieval system comprising: 
a computer; 
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a publish database accessible by said computer for storing a plurality 
of peer-reviewed articles; 
5 category information stored on said publish database together with 

each of the plurality of peer-reviewed articles; 

an update stored on said publish database together with a 
corresponding one of the plurality of peer-reviewed articles; 

a search request received by said computer from a user; and 
10 a program executing on said computer for retrieving an article and its 

corresponding update from said plurality of peer-reviewed articles in response 
to the search request. 

38. The research refrievaf system of etefm 35* farther Comprising a 
database of user profiles corresponding to users of the system. 

37. The research retrieval system of claim 36 wherein said user profiles 
include stored searches and folders. 

38. The research retrieval system of claim 36 further comprising a user 
identifier received by said computer, and wherein said program authenticates 
the user and retrieves a profile corresponding to the user from said user 
database. 

39. The research retrieval system of claim 38 further comprising a 
command received by said computer from a user for modifying the user's 
profile. 

40. The research retrieval system of claim 35 further comprising a letter 
received by said computer which pertains to a particular article on said publish 
database, and wherein the letter is peer-reviewed and stored on said publish 
database together with the particular article. 
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41 . The research retrieval system of claim 40 wherein said program 
retrieves letters pertaining to retrieved articles in response to a search 
request 

42. The research retrieval system of claim 40 wherein a payment is made 
by a user upon submission of the letter for peer review. 

43. The research retrieval system of claim 35 wherein an article from said 
publish database includes a description of supplies, and wherein hot links to 
information about the supplies are included in the article text upon payment of 
a fee by a manufacturer of the supply. 

44. A scholarly research conference organizing system comprising: 
a plurality of texts; 

a computer for receiving said plurality of texts; 

a database accessible to said computer for storing said plurality of 

5 texts; 

a program executing on said computer for forwarding a text notice to 
an organizer corresponding to a subject of each of the plurality of texts; 

a reviewer assignment input to said computer by the noticed organizer 
for each of the plurality of texts; and 
0 a score input to said computer by the assigned reviewer indicative of a 

level of interest in presentation of each of the plurality of texts at a conference. 

45. The conference organizing system of claim 44 wherein the 
corresponding organizer decides whether to present each of the plurality of 
texts at a conference with the aid of the reviewer score for the text. 

46. The conference organizing system of claim 45 wherein several 
reviewers are assigned to each text, and wherein the organizer makes the 
presentation decision with the benefit of several reviewer scores. 
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